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Because the disadvantages of technology 
implementation are not foreseen, they are often referred to as 
M unant icipated outcomes." Identifying side-effects is important 
because their negative effects are amplified when they arise without 
warning. This paper communicates the importance of unanticipated 
outcomes in educational technology settings, through a variety of 
examples collected over a decade of technology implementation 
evaluation in elementary and secondary educational settings. The 
Concerns Based Adoption Model (CBAM) indicates that teachers 
implementing an innovation are initially most concerned about their 
own interaction with the innovation and only later do they shift more 
of their concern to the students; this can result in many of the 
"teachable moments" being lost because the teacher was not part of 
the interaction. Because the teachers were often distanced from the 
computers while the students used them, their estimation of students' 
skills was often inaccurate, usually too high. Teachers who were 
distanced from their students' computer use were usually not aware of 
the amount of time the students spent on any one program. As a 
result, many students would switch computer programs as one might 
switch TV channels. Sometimes software features can be the source of 
negative side-effects, when students ignore the task at hand or the 
instructional material. (AEF) 
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1 Introduction 

According to Neil Postman (1995), “All technological change is a Faustian bargain. For every 
advantage a new technology offers, there is always a corresponding disadvantage.”(p. 193) 
Because the disadvantages are not foreseen, they are often referred to as “side-effects” or 
“unanticipated outcomes.” Identifying side-effects is important because their negative effects are 
amplified when they arise without warning. Strictly speaking, side-effects can be either positive 
or negative, but are usually seen as the latter. An exception to this rule comes from the 
experience of Michael Scriven (1974), who found “a product finished up in the ‘top ten’ in spite 
of zero results with respect to its intended outcomes because it did so well on an unanticipated 
effect.” (p. 34) Scriven (1991) writes, “Side-effects identification presents a methodological 
problem in that evaluators are usually cued to find the intended effects and their clients are usually 
much more interested in progress toward the intended effects.” (p. 331) His solution is “goal-free 
evaluation,” in which the evaluator is not informed about project goals until after the evaluation is 
completed, or nearly complete (Scriven, 1974). In current practice, side-effects may be lost for 
many reasons, such as a limited study design, the focus of the study on certain fixed dimensions, 
or the resulting reports omit those results outside of the “main stream” of interest. The purpose 
of this paper is to communicate the importance of unanticipated outcomes in educational 
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technology settings, through a variety of examples collected over a decade of technology 
implementation evaluation. 

Data Sources 

Since 1987, our research team has studied government policy and the adoption of computers in 
schools. We began when a suburban Toronto school board, having set a long-term goal of (a 
minimum of) three computers in each of its classrooms, asked us to evaluate a JK-8 pilot site. Our 
role in this study, observing classroom practice, led to a three-year investigation of the bilateral 
relationship between educational policy and practice, including the Ontario Ministry of Education, 
three Boards of Education, and more than a dozen classrooms. A variety of analyses have been 
earned out on different portions of the resulting textual database. 

Observed Outcomes 
Monitoring Student Use 

The Concerns Based Adoption Model, or CBAM (Hall, 1975) is one approach to implementing 
innovations and can serve as an organizer for this discussion. CBAM indicates that teachers 
implementing an innovation are initially most concerned about their own interaction with the 
innovation and only later do they shift more of their concern to the students. In one Third grade 
classroom, the teacher was asked how she monitored the activities of her students at the 
computers, to which she responded, “Oh, I never monitor what they do at the computers. I 
believe that whatever they do with the computers is important and valuable.” In a Second Grade 
classroom, we saw a teacher make several tours around the room, monitoring all of his students 
as they worked on a variety of tasks, but ignoring the two students who were using the 
computers. This was unfortunate, because they were consistently unsuccessful in copying text 
into the word processing program. After 45 minutes, one had a blank screen and the other had 
only a few words. These findings are compatible with those of Blackstock and Miller (1988), 
who observed 7-year old students using a variety of programs and noted that many of the 
“teachable moments,” times when the student-computer interaction had made the student ready 
for a new concept, were lost because the teacher was not part of the interaction. 



In contrast, we were also surprised to find two “new to classroom technology” teachers were 
very adept at identifying what their students were doing and what problems they were having. 
The insightful teachers didn’t have computers at home, but both of them had worked in business 
settings where computers were used (car rental and bank). This experience seemed to make a 
considerable difference in their ability to “see” what their students were doing. 

Distancing Teachers from Technology 

Both students and teachers were affected by the distancing of teachers from the technology. 
Because the teachers were often distanced from the computers while the students used them, their 
estimation of the students’ skills was often inaccurate, usually too high. On the other hand, 
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distancing provided an opportunity for students to be virtual experts. In a first grade class, the 
teacher told us that one boy was the class expert in using the computers. A few moments later, 
the boy was called over to a computer where a fellow student was having trouble. The expert 
tried several keys, but nothing worked and he drifted away. A short time later, the problem was 
resolved by the student using the computer, but on seeing this, the “expert” informed the teacher 
that he had fixed it and the teacher accepted his claim at face value. 



One should not assume however, that all primary students were deficient in their computer 
skills. For example, in a Kindergarten class, one student not only gave us a complete and 
informative overview of the software we were attempting to use, but he also came back later to 
make sure we weren’t having any problems. In another classroom, when a boy was talking to me 
about his preferences for two programming languages, BASIC and Logo, it was difficult for me 
to remember that this student was only in the second grade. One first grader had learned the 
process of animation from watching older boys using computers in the library and became an 
instructor on animation for his class. Teachers have also used the concept of local expert to good 
advantage in writing classes, with each student being a source of expertise on one topic, such as 
quotation marks, or formatting. 



Channel Hopping and TV 

The computer behavior of teachers and students was in accordance with their own agendas, more 
so as the distance increased. Teachers who were distanced from their students’ computer use 
were usually not aware of the amount of time the students spent on any one program. As a result, 
the students would “channel hop,” switching computer programs as one might switch TV 
channels. As teachers became more adept at recognizing what their students were doing at the 
computers, they often adopted a simple tactic, permitting students to use a computer only when 
they were able to state a goal and only as long as they continued toward that goal. If the student 
changed to another program, the teacher would assign the computer to another student. 

Of course, not all students tried to channel hop at the computers. Some were very task- 
oriented and completed one task before moving to anything else. On one occasion, I interviewed 
two students with vastly different “viewing habits.” Not too surprisingly, the student who channel 
hopped in the classroom, was unable to tell me the complete story from a recently viewed TV 
program, while the other student was more disciplined both at the computer and with TV, giving 
me a detailed description of the plot from a recent program. These observations suggest that links 
between computer and TV behavior might be worth exploring, particularly when viewed in the 
context of Kubey and Csikszentmihalyi’s (1990) findings on TV and the quality of life. 

Software Features 

We often assume that more features make for better software, but sometimes the features interfere 
with the intended application. That is, the features can become the source of negative side-effects. 
On several occasions, we noticed upper-elementary students who were not only using an ornate 



gothic script for their font, but had also increased the character size so that only a few words filled 
the screen. In one instance, an argument developed over the spelling of a word because one 
student couldn’t recognize the letters in gothic script. 

While considering writing, it is worth noting that the side-effects associated with using word 
processing, at least among adults, seem to be very personal, suggesting the presence of what 
might be called “aptitude-feature interactions” (AFI). For example, a number of people have 
reported a problem in entering a first draft into the word processor. These people are unable to 
pause in their typing without shifting into “revision mode” and interrupting the flow of their first 
draft. Some find the act of deleting text to be painful, while others have never experienced either 
problem and are unable to understand the difficulty. Exploration of these and other AFI’s is 
another promising research avenue. 

Agendas and Features 

AFI might also stand for “Agenda- Feature Interaction,” as in a writing application, electronic mail, 
which was a disaster at a school that implemented an internal system. In order to send a message, 
the student had to first create a message file using the word processor, then switch to the mail 
program to send the file. Unfortunately, students were more interested in sending messages than 
they were in writing them. As a result, they used the mail program to send whatever files were in 
their directory, such as drawings, summaries of math quiz results, etc. The mail system soon 
became clogged with this “junk mail” and on the few occasions when someone did try to send a 
“real message,” they were often unsuccessful because the mail system had no more space for 
messages. 



In contrast, email between schools, particularly between countries, has been very productive in 
enhancing writing skills by giving meaning to the communications aspects of writing. One reason 
may be that students are less likely to send junk mail to students they don’t know, but the more 
likely explanation is that a real writing task, one where information is exchanged, is more 
motivating. 

Students are often very skilled at ignoring the instructional material in educational software. In 
the “Math Maze” program, some students discovered that you could progress through the maze 
by only using the return key. It took many repetitions of this key, but eventually the software 
would give the student the answer to the problem and allow him to proceed through the maze. 
“Cargo Sailer” was a favorite with many students who ignored the stated task of delivering cargo 
to a specified (by latitude and longitude) port in favor of sailing aimlessly until the food ran out 
and the crew starved. These students were satisfied to experience the storm scene, complete with 
howling winds, and other special effects, returning to the program time after time for this reward. 
They knocked the “edu” out of “edutainment.” 
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A series of historical simulation programs all used the same format: the student would be 
presented with a social, political, or economic problem and given 3-4 choices from which to 
choose. Students spent considerable time studying the problem and the options (time for me to 
read through it at least three times), but spent little time (too short for me to read it) reading the 
feedback explaining the positive and negative aspects of their choice. In other words, they spent 

hardly any time with the part of the program that did the most teaching, leaving only the 
“tainment.” 



In contrast, a seventh grade girl, on the verge of being assigned to special education, found in 
these simulations an environment in which she could excel. She would rehearse her skills by 
repeating these programs until she mastered all the component tasks, as she proudly demonstrated 
for me. Similarly, a special education teacher spoke in glowing terms about a simple program that 

had initially annoyed him, until he discovered that “his guys” could experience success with this 
program. 

Gender and Features 

The discussion would not be complete without a discussion of “Gender-Feature Interaction” 
(GFI). Boys are often competitive and their educational use of computers was no exception. Two 
boys seated at adjacent computers were entering text into word processors. Their attention was 
directed at the counter indicating how many lines of text had been entered. They entered large 
portions of short lines of text, and also many blank lines, as each sought to enter more lines than 
his competitor. The historical simulations also fed into this competitive streak, providing a 
multidimensional “score” for each stage, and the boys again focused on the numbers, arguing that 
the dimension on which they had the highest score was the most relevant. 

These specific findings may go out of date quickly for a number of reasons, but many of the 
underlying principals are probably alive and well (Ragsdale, 1988). It would not be surprising to 
find that students are still skilled at isolating the instructional component from the entertainment. 
It is also quite likely that students’ habits from their out of school activities, particularly TV, are 
also a big factor in their school use of computers. As we continue to increase the use of 
technology in classrooms, we must continue to seek out and identify the unanticipated outcomes, 
so that we can refine our approach to technology and thereby enhance our students’ benefits from 
technology. Our goal should be to change beneficial unanticipated outcomes to anticipated 
outcomes, while changing the negative unanticipated outcomes to unrealized outcomes. 
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